5 results
8 Detection of Feigned ADHD through an Experimental MMPI-2 ADHD Validity Scale among U.S. Military Veterans
- Christopher T. Burley, Timothy J. Arentsen, Jennifer S. Seeley McGee, Katie M. Califano, Holly R. Winiarski, Marcy C. Adler, Brad L. Roper
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, pp. 693-694
-
- Article
-
- You have access Access
- Export citation
-
Objective:
The prevalence of ADHD diagnoses more than doubled in VA settings between 2009 and 2016 (Hale et al., 2020). However, attentional difficulties are not exclusive to ADHD and can also be seen in non-neurodevelopmental disorders, including depression, anxiety, substance use, and PTSD (Marshall et al., 2018, Suhr et al., 2008). Further, patients can easily feign symptoms of ADHD with few available instruments for accurate detection (Robinson & Rogers, 2018). Given the significant symptom overlap and rising rates of reported ADHD among Veterans, accurate detection of feigned ADHD is essential.
This study examined the utility of the experimental Dissimulation ADHD scale (Ds-ADHD; Robinson & Rogers, 2018) on the MMPI-2, in detecting feigned ADHD presentation within a mixed sample of Veterans.
Participants and Methods:In this retrospective study, 173 Veterans (Mage = 36.18, SDage = 11.10, Medu = 14.01, SDedu = 2.11, 88% male, 81% White, and 17% Black) were referred for neuropsychological evaluation of ADHD that included the MMPI-2 and up to 10 PVTs. Participants were assigned to a credible group (n=146) if they passed all PVTs or a non-credible group (n=27) if they failed two or more PVTs. Group assignment was also clinically confirmed. The Ds-ADHD was used to differentiate groups who either had credible or non-credible performance on cognitive measures. Consistent with Robinson and Rogers’ study, “true” answers (i.e., erroneous stereotypes) were coded as 1 and “false” answers were coded as 2, creating a 10- to 20-point scale. Lower scores were associated with a higher likelihood of a feigned ADHD presentation.
Results:Preliminary analyses revealed no significant group differences in age, education, race, or gender (ps > .05). An ANOVA indicated a significant difference between groups (F[1, 171] = 10.44, p = .001; Cohen’s d = .68) for Ds-ADHD raw scores; Veterans in the non-credible group reported more “erroneous stereotypes” of ADHD (M raw score = 13.33, SD = 2.20) than those in the credible group (M = 14.82, SD = 2.20). A ROC analysis indicated AUC of .691 (95% CI = .58 to .80). In addition, a cut score of <12 resulted in specificity of 91.8% and sensitivity of 18.5%, whereas a cut score of <13 resulted in specificity of 83.6% and sensitivity of 44.4%.
Conclusions:The Ds-ADHD scale demonstrated significant differences between credible and non-credible respondents in a real-world setting. Previously, this scale has primarily been studied within laboratory settings. Further, results indicate a cut score of <12 could be used in order to achieve adequate specificity (i.e., >90%), which were similar findings to a study examining SVT-based groups (Winiarski et al., 2023). These results differ slightly from prior research by Robinson and Rogers (2018), who indicated a cut score of <13 based on the initial simulation-based study. In similar clinical settings, where there are high rates of psychiatric comorbidity, a cut score of <12 may prove clinically useful. However, this cut-score was associated with low sensitivity within this mixed Veteran sample. Further research should focus on replicating findings within other clinical settings, including ones with larger non-credible samples.
16 Validation of the Pillbox Test in a Veteran Sample: A Replication Study.
- Frances M. Bozsik, Timothy J. Arentsen, Brad L. Roper, Jessica M. Fett, Marcy C. Adler, Jennifer L. Jacobson, Charlotte Bayer
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, pp. 699-700
-
- Article
-
- You have access Access
- Export citation
-
Objective:
Assessment of medication management, an instrumental activity of daily living (IADL), is particularly important among Veterans, who are prescribed an average of 2540 prescriptions per year (Nguyen et al., 2017). The Pillbox Test (PT) is a brief, performance-based measure that was designed as an ecologically valid measure of executive functioning (EF; Zartman, Hilsabeck, Guarnaccia, & Houtz, 2013), the cognitive domain most predictive of successful medication schedule management (Suchy, Ziemnik, Niermeyer, & Brothers, 2020). However, a validation study by Logue, Marceaux, Balldin, and Hilsabeck (2015) found that EF predicted performance on the PT more so than processing speed (PS), but not the language, attention, visuospatial, and memory domains combined. Thus, this project sought to increase generalizability of the latter study by replicating and extending their investigation utilizing a larger set of neuropsychological tests.
Participants and Methods:Participants included 176 patients in a mixed clinical sample (5.1% female, 43.2% Black/African American, 55.7% white, Mage = 70.7 years, SDage = 9.3, Medu = 12.6 years, SDedu = 2.6) who completed a comprehensive neuropsychological evaluation in a VA medical center. All participants completed the PT where they had five minutes to organize five pill bottles using a seven-day pillbox according to standardized instructions on the labels. Participants also completed some combination of 26 neuropsychological tests (i.e., participants did not complete every test as evaluations were tailored to disparate referral questions). Correlations between completed tests and number of pillbox errors were evaluated. These tests were then combined into the following six domains: language, visuospatial, working memory (WM), psychomotor/PS, memory, and EF. Hierarchical multiple regression was completed using these domains to predict pillbox errors.
Results:Spearman’s correlation coefficients indicated that 25 tests had a weak to moderate relationship with PT total errors (rs = 0.23 -0.51); forward digit span was not significantly related (rs = 0.13). A forced-entry multiple regression was run to predict PT total errors from the six domains. The model accounted for 29% of the variance in PT performance, F(6, 169) = 11.56, p < .001. Of the domains, psychomotor/PS made the greatest contribution, f(169) = 2.73, p = .007, followed by language, f(169) = 2.41, p = .017, and WM, f(169) = 2.15, p = .033. Visuospatial performance and EF did not make significant contributions (ps>.05). Next, two hierarchical multiple regressions were run. Results indicated that EF predicted performance on the PT beyond measures of PS, AR2 = .02, p = .044, but not beyond the combination of all cognitive domains, AR2 = .00, p = .863.
Conclusions:Results of this study partially replicated the findings of Logue et al. (2015). Namely, EF predicted PT performance beyond PS, but not other cognitive domains. However, when all predictors were entered into the same model, visuospatial performance did not significantly contribute to the prediction of pillbox errors. These results suggest that providers may benefit from investigating medication management abilities when deficits in PS, WM, and/or language are identified. Further research is needed to better understand which domains best predict PT failure.
20 Clinical Utility of an Experimental Ds-ADHD Validity Scale in Detection of Feigned ADHD symptoms in a U.S. Military Population
- Holly R Winiarski, Timothy J Arentsen, Marcy C Adler, Christopher T Burley, Katie M Califano, Jennifer S Seeley-McGee, Brad L Roper
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, p. 703
-
- Article
-
- You have access Access
- Export citation
-
Objective:
Accurate identification of Attention-Deficit/Hyperactivity Disorder (ADHD) is complicated by possible secondary gain, overlap of symptoms with psychiatric disorders, and face validity of measures (Suhr et al., 2011; Shura et al., 2017). To assist with diagnostic clarification, an experimental Dissimulation ADHD scale (Ds-ADHD; Robinson & Rogers, 2018) on the MMPI-2 was found to distinguish credible from non-credible respondents defined by Performance Validity Test (PVT)-based group assignment in Veterans (Burley et al., 2023). However, symptom and performance validity have been understood as unique constructs (Van Dyke et al., 2013), with Symptom Validity Tests (SVTs) more accurately identifying over-reporting of symptoms in ADHD (White et al., 2022). The current study sought to evaluate the effectiveness of the Ds-ADHD scale using an SVT, namely the Infrequency Index of CAARS (CII; Suhr et al., 2011), for group assignment within a mixed sample of Veterans.
Participants and Methods:In this retrospective study, 187 Veterans (Mage = 36.76, SDage = 11.25, Medu = 14.02, SDedu = 2.10, 83% male, 19% black, 78% white) were referred for neuropsychological evaluation of ADHD and administered a battery that included internally consistent MMPI-2 and CAARS profiles. Veterans were assigned to a credible group (n=134) if CII was <21 or a non-credible group (n=53) if CII was >21. The Ds-ADHD scale was calculated for the MMPI-2. Consistent with Robinson and Rogers (2018), “true” answers (i.e., erroneous stereotypes) were coded as 1 and “false” answers were coded as 2, creating a 10- to 20-point scale. Lower scores were associated with a higher likelihood of a feigned ADHD presentation.
Results:Analyses revealed no significant differences in age, education, race, or gender (ps > .05) between credible and non-credible groups. An ANOVA indicated a significant difference between groups (F[1,185] = 24.78, p <.001; Cohen’s d = 0.80) for Ds-ADHD raw scores. Veterans in the non-credible group reported more “erroneous stereotypes” of ADHD (M raw score = 13.23, SD = 2.10) than those in the credible group (M = 14.94, SD = 2.13). A ROC analysis indicated AUC of .72 (95% CI = .64 to .80). In addition, a Ds-ADHD cut score of <12 resulted in specificity of 94.5% and sensitivity of 22.6%, whereas a cut score of <13 resulted in specificity of 85.8% and sensitivity of 50.9%. When analyzing other CII cut scores recommended in the literature, results were essentially similar. Specifically, analyses were repeated when group assignment was defined by cut score of CII<18 and by removing an intermediate group (CII = 18 to 21; n=24).
Conclusions:The Ds-ADHD scale demonstrated significant differences between credible and non-credible respondents in a Veteran population. Results suggest a cut score of <12 had adequate specificity (.95) with low sensitivity (.23). This is consistent with findings using PVTs for group assignment that indicated a cut score of <12 had adequate specificity (.92) with low sensitivity (.19; Burley et al., 2023). Taken together, findings suggest that the Ds-ADHD scale demonstrates utility in the dissociation of credible from non-credible responding. Further research should evaluate the utility of the scale in other clinical populations.
31 The ADHD Dissimulation Scale (Ds- ADHD) on the MMPI-2-RF versus Established MMPI-2-RF Validity Scales
- Katie M Califano, Timothy J Arentsen, Holly R Winiarski, Christopher T Burley, Marcy C Adler, Jennifer S Seeley McGee, Brad L Roper
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, pp. 712-713
-
- Article
-
- You have access Access
- Export citation
-
Objective:
The MMPI-2-RF contains scales that assess different types of invalid response styles, especially potential symptom over-reporting (e.g., F-r, Fs, Fp-r, FBS-r, RBS). However, these scales are not designed to specifically capture noncredible symptoms reports associated with Attention-Deficit/Hyperactivity Disorder (ADHD). Robinson & Rogers (2018) proposed the experimental Dissimulation ADHD validity scale (Ds-ADHD) on the MMPI-2-RF that was effective in distinguishing credible and non-credible ADHD diagnoses via a simulator-based study. Within the current study, the Ds-ADHD scale was compared to the established MMPI-2-RF validity scales within a mixed sample of U.S. Military Veterans.
Participants and Methods:173 Veterans (Mage = 36.18, SDage = 11.10, Medu = 14.01, SDedu = 2.11, 88% male, 81% White, 17% Black) completed a neuropsychological evaluation which included an internally consistent MMPI-2-RF profile and up to 10 performance validity tests (PVTs) as well as a question about a possible ADHD diagnosis. The credible group was determined if participants passed all PVTs (n=146) and completed at least 2 PVTs. The non-credible group was determined by failing two or more PVTs (n=27). Group assignment was clinically confirmed. The Ds-ADHD scale was calculated according to Robinson & Rogers’ (2018); responses of “true” (i.e., erroneous stereotypes) were coded as 1 and “false” answers were coded 2, creating a 10- to 20-point scale. Thus, lower scores would be associated with a higher likelihood of a feigned ADHD presentation. Other MMPI-2-RF validity scales of interest included F-r, Fs, Fp-r, FBS-r, and RBS.
Results:The established MMPI-2-RF validity scales were significantly correlated with PVT group membership, but correlations were weak to moderately strong (rS ranged from -.43 to -.18; ps < .05). A series of stepwise regression models were completed with the Ds-ADHD scale and one of the MMPI-2-RF validity scales as independent variables, with group membership as the dependent variable. Ds-ADHD) contributed uniquely to each model (CÜ ranged from .03 to .04, ps < .05). The established MMPI-2-RF validity scales effectively classified group membership (AUC values ranged from .57 to .68), and the Ds-ADHD scale had a marginally higher AUC (.69); however, it was not statistically significantly stronger than any of the established scales (ps > .05).
Conclusions:Clinicians interested in identifying potentially simulated ADHD presentations with the MMPI-2-RF may desire to calculate the Ds-ADHD scale, which previously only had support from a simulator-based study. The Ds-ADHD scale significantly contributed to each model, suggesting that it helped explain groups over and above each of the traditional MMPI-2-RF validity scales. However, it only had a marginally stronger ability to classify participants, indicating that there may be diminishing returns for clinicians. Among the traditional validity scales, RBS and F-r best classified groups, and FBS-r was the least effective. This study employed a cross-sectional design in a mixed sample of Veterans undergoing a neuropsychological evaluation. Future research should focus on replicating the findings using a credible sample that was limited to an independently verified diagnosis of ADHD.
84 Using a Combination Score from the Full Rey Complex Figure Test (RCFT) as a Performance Validity Measure (PVT)
- Jessica M. Fett, Timothy J. Arentsen, Brad L. Roper
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, pp. 757-758
-
- Article
-
- You have access Access
- Export citation
-
Objective:
Some RCFT indices are effective Performance Validity Test (PVTs) during neuropsychological evaluations. A combination score that includes the copy score, true positive recognition, and atypical errors has proven to be especially useful (see Lu et al, 2003). However, this score was derived from administration that deviated from protocols outlined by Meyers & Meyers (1995) in that the Recognition trial was administered after the 3-minute delay instead of the 30-minute delay. The current study examined the utility of the RCFT combination score as a performance validity test (PVT) when completing the recognition trial after the 30-minute delay.
Participants and Methods:This study utilized archival data from 298 Veterans who presented for a clinical neuropsychological evaluation at a southern Veterans Affairs Medical Center. The evaluation included up to nine PVTs and all trials of the RCFT (per Meyers & Meyers, 1995). Patients were considered credible if all PVT performance fell within normal limits. This resulted in 232 patients in the credible group (Mage = 52.9 years, SDage = 15.2, Medu = SDedu = 2.5, 88% male, 71.2% White, 28.3% Black/African American). Patients were considered non-credible if they failed >2 PVTs. This resulted in 66 patients in the non-credible group (Mage = 51.6, SDage = 13.79, Medu = SDedu = 2.4, 92.4% male, 56.1% White, 43.9% Black/African American). Group assignment was also clinically confirmed. Receiver operating characteristic (ROC) curve analyses were conducted to discriminate between credible and non-credible groups utilizing the established RCFT combination score.
Results:RCFT combination scores distinguished groups, with credible participants scoring higher than non-credible participants (F[1, 296]=63.76, p<.001, d=1.11; M = 56.9, SD = 9.3 vs. M = 46.5, SD = 9.5, respectively). A ROC analysis indicated AUC = .800 (95% CI = .73 to .86). When specificity was set at >90%, a cut-score of <46.5 yielded sensitivity at 46.0%. The analogous cut-score from the Lu et al. (2003) study (i.e., <47) was associated with a specificity of 88.7 and sensitivity of 46.0% in the current study.
Conclusions:As the Lu et al. (2003) established the combination score of the RCFT with procedures that deviated from the standardized protocol outlined by Meyers and Meyers (1995), clinicians who opted to adhere to Meyers and Meyers’ full protocol may have concerns about using the combination score as a PVT. The current study established a similar cut-off score to what Lu et al., (2003) reported (i.e., <46.5 vs. <47) while following a different administration procedure of the RCFT. Also, the index was moderately sensitive in the current study (i.e., 45.5%) but less so than what Lu et al. reported when using a cut-score that had >90% specificity (i.e., 75.9% sensitivity). This suggests that the index may be robust to deviations in administration procedures. Difference in sensitivity could be related to difference between samples. As the current sample was derived from a clinical, VA setting, current findings extend the generalizability of the index. Future research would benefit exploring if any subgroups would benefit from adjusted cut-scores to reduce the risk of false positive identification.